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Abstract. This study examines the effectiveness of an automated translation chatbot 
used in online interactions which consequently could enhance second/foreign 
language (L2) competence. Based on the sociocultural perspectives of learning, 
such as communication to recognize the difference from others and to be involved 
in sense-making processes, this study examines the automated translation chatbot 
to translate L1 statements into L2 automatically during online interactions by 
hypothesizing that the chatbot provides a variety of L2 comprehensive input and 
lowers learners’ anxiety to write their L2 posts, which will lead to successful L2 
learning. To verify our hypothesis, quantitative and qualitative data was collected 
by the online interaction, essay writing tasks, and open-ended questionnaire before 
and after the interaction. The findings of this study will suggest that the efficient 
use of an online translation bot facilitates collaborative dialog and results in more 
successful L2 learning. 


Keywords: collaborative learning, translation chatbot, online learning community, 
comprehensive input, incidental learning. 

1. Introduction 

Collaborative activities have been acknowledged as being beneficial for 


L2 learning. This is because of the perspective that these activities serve as 
communication to recognize the differences from others and establish shared 
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meanings (Sharples, 2005). Additionally, such activities could foster learner 
autonomy particularly when the tasks were conducted via mobile devices (e.g. 
Reinders, 2011; Sato, Murase, & Burden, 2015). This stems from their nature, 
which enables prompt access to resources and feedback outside the classroom 
and consequently facilitates out-of-class learning, which can supplement in-class 
activities (Kukulska-Hulme, 2015). 


However, some challenges are left for mobile-based L2 collaboration. Stockwell 
and Hubbard (2013) showed that reading and writing L2 texts on small 
screens could interfere with learning. Writing L2 texts, in particular, would be 
bothersome for learners who have not often typed the alphabet with their mobile 
devices. Learners’ anxiety for the accuracy of their L2 output can also be another 
interference. McCarty, Obari, and Sato (2017) reported that Japanese L2 learners’ 
inactivity in an online collaboration results mainly from their perception that 
their L2 competence is not sufficient enough for their peers to understand their 
posts. These might prevent their interaction with others and yield unsuccessful 
L2 learning. 


This study aims to tackle these challenges and to facilitate collaborative activities 
for successful L2 learning. This study utilizes a function in an online communication 
app: an automated translation chatbot. This bot is offered by LINE, one of the most 
popular online messenger apps in Japan. It can be used when an online interaction 
is held using LINE and automatically generates an L2 translation of a statement 
posted by users. Therefore, the users can see any statement both in L1 and L2 
simultaneously on the screen (see Figure 1). 


The chatbot can be used during online interactions between peers or among a group, 
while other machine translation applications like Google Translation cannot be 
used during the interaction because their L2 output is not provided automatically, 
interfering the interaction with peers as a result. This is the difference between our 
study and other studies applying machine translations to L2 learning (e.g. Garcia 
& Pena, 2011). 


The bot offers various types of L2 translation together with the L1 texts, some 
of which L2 learners could not generate on their own. In that sense, L2 learners 
are exposed to comprehensive input (Krashen, 1982). Besides, the use of the bot 
reduces learners’ inferiority about their L2 competence. These advantages lead our 
study to hypothesize that the chatbot will scaffold online interactions, leading to 
the enhancement of L2 performance and motivation for collaborative learning by 
the following research questions: 
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¢ RQlI: Could L2 learners doing collaborative learning with the translation 
bot retain the L2 words or expressions found in the automated output? 


¢ RQ2: Could L2 learners produce sophisticated L2 sentences after the 
collaborative learning with the bot? 


¢ RQ3: Did the experience of the bot change their attitude towards 
collaborative tasks? 


Figure 1. An example of an online interaction with the chatbot 


| PEKOM 

HOMES SRWKS | ED 
CWTWSBAMBWOCS 
ho HRERDX OAM 
RbATREOKNE RTH 
HC SRORETRER 
cRNSATCS Ma? 


Thank you!There are 
certainly many similarities. |f 
all citizens could act on the 
assumption that individual 
humans are the most 
important thing, would they 


be truly a democracy? 

_ 2407 
EDIcZSECRBWEST. H 
REarts BASE 
, WRERSOATCRWT 
L&Do 


| think so.If everyone thinks 
so, autocracy and 


dictatorship will not emerge. 


Tel 


2. Method 


2.1. Participants and the activity 


Three undergraduates at a university in Japan attended the activity; all of them are 
freshmen but from different departments. They were invited to the research: once 
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a week, they read a book related to political issues and exchanged their opinions in 
a classroom and then conducted supplemental discussions of the issues via LINE 
with their mobile devices until the next class. 


2.2. Research procedures 


After listening to the description of the research project, the participants were asked 
to answer an open-ended questionnaire regarding the attitude towards collaborative 
learning. Then they worked on an essay writing task using Criterion®, an essay 
writing evaluation system by the Test of English as a Foreign Language (TOEFL"). 
The essay topic we chose from Criterion was a political issue similar to the book 
they read. Moreover, a questionnaire was conducted which consists of 34 items to 
measure learners’ belief for collaboration in terms of usefulness of cooperation, 
individual orientation, and inequity (Nagahama, Yasunaga, Sekita, & Kouhara, 
2009). 


The group discussion via LINE, which was automatically translated into English, 
was conducted as a supplementary task related to the in-class activities. The 
participants could freely post their opinions about the materials they read or answer 
the questions posted by other participants and an author of the present study. After 
about one month of online interactions, a vocabulary test and follow-up interviews 
were administered as well as the same questionnaire and writing task. The words 
for the test were chosen from the L2 sentences the chatbot generated that the 
participants might not have known before the activity. The follow-up interviews 
were conducted to confirm the change of their attitude for this activity. 


All the data derived from L2 translations, the vocabulary test, two writing tasks, 
and two questionnaires was collected and then the improvement between the 
online discussion with the chatbot was observed. However, no statistical analysis 
was implemented due to the small number of participants. 


3. Findings 


As a whole, few positive changes were observed: low score of the vocabulary test 
(RQ1); no improvement of their essay in terms of the evaluation score, and three 
items of Criterion® (RQ2), but their motivation toward collaborative learning 
was enhanced according to the questionnaire (RQ3). The follow-up interviews 
underlay these findings. For example, Student #3, who got the highest score in 
the vocabulary test and used the expression generated by the chatbot in his/her 
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essay, revealed that the student carefully read the translated output (“a couple of 
Japanese words with different connotations are translated into the same word in 
English”). The student also noticed that the exposure to the translation potentially 
triggered him/her to use some of the words. The interview with Student #3 also 
indicated that the student evaluated the quality of L2 translation and his/her 
learning process (“The translation had some mistakes, which I thought I should 
avoid making”). 


Meanwhile, Student #2 rarely read analytically the translated output. The student 
also said that he/she did not remember what words he/she encountered in the chat 
space and stated that these words need to be encountered many times to be learned. 
We interpret that the difference between these two students is two-fold. First, 
Student #3 consciously evaluated his learning, which was not the case with Student 
#2. The second point is that while Student #3 tried using some of the encountered 
words, Student #2 did not have such an attempt in the essay. However, it was 
found that both students shared a recognition that collaborative learning using the 
translation chatbot can work positively. 


4. Discussion and conclusion 


This study aimed to verify the effectiveness of an automated translation chatbot 
for L2 learning. McCarty et al. (2017) showed that the L2 sentences written by 
the participants using the chatbot during one month of collaborative tasks became 
more elaborate than the sentences before the task. The current study, however, 
showed little improvement of the learners’ L2 performance in terms of word recall 
(RQ1) and writing (RQ2), although their belief towards collaborative learning 
was positively changed (RQ3). The findings could not support the theoretical 
advantages of collaborative L2 learning with the automated chatbot in terms of L2 
proficiency. However, the chatbot might help the learners to make their belief of 
collaborative learning more positive. 


It should be noted that any generalized conclusion could not be derived from our 
current study due to the limited number of participants, with no control group 
available. However, this study might indicate some pedagogical implications to 
utilizing the chatbot for L2 learning practices: instructors should choose discussion 
topics for participants to be more involved in the discussion; some tasks can be 
implemented to examine whether or not the L2 translations by the chatbot could 
translate L1 statements correctly. To devise pedagogical use of the chatbot, further 
studies should be conducted. 
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